I'm not very good at C, so I had no idea how to constrict most of the code here to be only run on a Linux system as I know that will be a big problem as if this is merged it means that no other platform will be able to build.

I guess this is "BTLE" (BLE) described here:

https://support.apple.com/guide/deployment/use-airplay-dep9151c4ace/web

https://www.bluetooth.com/bluetooth-resources/the-bluetooth-low-energy-primer/

https://www.allaboutcircuits.com/technical-articles/exploring-the-basics-of-bluetooth-low-energy-a-beginners-guide-to-ble/

https://github.com/microsoft/windows-universal-samples/tree/main/Samples/BluetoothAdvertisement

it's present in windows and macOS too.

• so uxplay needs to detect if BLE service is present on the host, and if so, send a DNS_SD-equivalent advertisement periodically.
  will the reply come over bluetooth too, on on the regular network? (I didnt read the doc yet)

That's it, Apple describes it as the "Bluetooth IP Address Advertisement". BLE avertisements can be sent out on Windows and macOS too - I might have a go at implementing it on Windows later, I'll just need to take a look at existing code and figure out how one implements different libraries for each platform etc. as I've never worked in C before.

• so uxplay needs to detect if BLE service is present on the host, and if so, send a DNS_SD-equivalent advertisement periodically.

will the reply come over bluetooth too, on on the regular network? (I didnt read the doc yet)

Currently, the code is set so that it attempts to set up a BLE interface, if it doesn't find one/an error occurs (usually that it doesn't have the privileges as the program needs to be run as sudo) then it sends a message in the console saying it is disabled for the session as there was an error and gives up using BLE. It doesn't exactly send a DNS-SD advertisement over bluetooth, the payload size of a BLE advertisement is very small due to power limitations and so instead all that gets sent in the payload is the following 8 bytes:

09 06 03 - Represents an AirPlay device
XX - A single byte that is used as a state machine, if the state of the AirPlay device changes (the password changes, or if the name changes, or if the availability of AirPlay goes from Everyone -> Anyone on the same network etc. etc. ) the byte increments by one indicating that the sender should repoll for the txtAirplay and txtRAOP records again.
c0 a8 01 XX - The IP address that the sender should connect to to retrieve the txtAirplay and txtRAOP records.

All that still needs to be implemented is using the Windows BLE library if on Windows and I assume using the linux library on mac (unless there is an alternative - Will need to look into this). Also I've just realised that it only gets the ethernet iface IP address atm, so people using Wi-Fi won't have their ip broadcast - I will fix this soon.

can you take a look at SimpleBLE (a GPL v3 cross-platfom BLE implementation) ?
Could this be used?

https://github.com/OpenBluetoothToolbox/SimpleBLE

Unfortunately the large number of "issues" suggests its not quite mature yet....

... as I've never worked in C before.

C++ with "extern C" wrappers for interface with C works. uxplay.cpp does this.

I remember seeing this library when I began trying to put it all together, since I didn't really know anything about C I stayed away from it because of the no. of issues (fixing my own problems was going to be enough, let alone having problems because the library was causing them). Will take a look at it tonight and try to use this library instead as that will make it a lot easier than implementing it differently for each platform.

BLE does look like a good feaure to add, thanks for your work on this

Thank you - Off topic but I've been fighting for a while to try to figure out how H265 works, struggling with wrapping my head around the HomeKit pairing process as it's not easy. Will keep trying though as I'm keen on figuring it out. Whilst I've got you - How do you reverse engineer all of this? I'm sure at the start you could use existing docs and research, but the things you're doing currently like HLS aren't really that well documented, and so I'm interested to know how you figure the stuff like that out when it's behind an encrypted channel of communication? Do you have a custom proxy solution that you use?

HLS was a total mystery. incomplete pieces of a code with no ancestry in our own code base had appeared on github from time to time, with no license or provenance., until late last year, when someone interested noticed the full C++ code was recently published with a GPL license as "apsdk-public", but not in a usable state, as a demo and the fairplay substitute (playfair) were stripped out. It seems that a single author had been working on it since 2017. Luckily, looking back in the code history we were able to restore an earlier working demo, and see how to put back fairplay, which was indeed what was being used. (It's the only partial "crack" of fairplay that has been achieved, so I believe all open source and non-apple closed source implementations use it). Finally, by filling up the working demo with printf statements at every function call, the HLS workings became traceable and understandable. A working example that one can add printf's to is the key, when there is not much documentation.

apsdk-public has a C++ interface to a 2017 C hlsparser found on github. we borrowed this interface as apsdk is GPL, and gave C wrappers to it, and connected it to adapted UxPlay. . The remaining issues are GStreamer integration, so at some point uxplay will get HLS support (at least for youtube videos, since netflix has removed non-apple airplay support.)

Homekit pairing (from the client viewpoint) is semi documented at https://github.com/ejurgensen/pair_ap with server and client implementation. This also includes a client-only implementation of legacy-pairing it calls "fruit pairing" with SRP that was very useful for using on a true AppleTV to see what its SRP server responses were (this is the one-time pairing protocol, also used in homekit) , and fill in undocumented details by guess-and-check. https://github.com/philippe44/libraop was also very useful, but does not do homekit.

see https://github.com/FDH2/UxPlay/wiki/AirPlay2-protocol

I'm interested to know how you figure the stuff like that out when it's behind an encrypted channel of communication? Do you have a custom proxy solution that you use?

Unfortunately not. In legacy pairing (even with SRP authentication, which sets up what COULD be used for total communication encryption ) I was surprised to find that after all the work to implement SRP authentication, it doesnt enforce total encryption. But homekit does, as can been seen in https://github.com/ejurgensen/pair_ap, and seems to use (apple-modified) SRP with the "big random integer" upped from 2048 to 3072 bits.

09 06 03 - Represents an AirPlay device
XX - A single byte that is used as a state machine, if the state of the AirPlay device changes (the password changes, or if the name changes, or if the availability of AirPlay goes from Everyone -> Anyone on the same network etc. etc. ) the byte increments by one indicating that the sender should repoll for the txtAirplay and txtRAOP records again.
c0 a8 01 XX - The IP address that the sender should connect to to retrieve the txtAirplay and txtRAOP records.

one design possibility is that the running uxplay writes its BLE message to a file that only exists while uxplay is running, and some helper function, running as root if that is needed, broadcasts it on bluetooth at appropriate intervals if it exists? something like that is implement in the -dacp option for interfacing with a remote controller for the iOS client.

I am assuming that the client responds on the network, not with reverse BLE?

To keep it simple there are two advertising types; connectable and non-connectable. Non-connectable might be useful if you just need one-way communication and only need to broadcast a couple of bytes. One use case for this might for example be location awareness.

• Are we talking about the connectable or non-connectable case in the above quote about BLE?

I think apple generally uses the non-connectable BLE, for things like

• airpod battery ads
• Apple TV ads: (screen calibration, apple id login)

e.g https://github.com/MarketStreetCyber/Blabber/blob/main/impersonate/setup_appletv.py uses "ADV_NONCONN_IND" which I assume is non connectable

SimpleBLE won't be enough unfortunately - It doesn't seem to support broadcasting advertisements (and it took me 4 hours to include in the project). I'll have to have a go at writing an implementation for each platform.

We're talking about non-connectable here, as @thiccaxe said the ADV_NONCONN_IND. Meaning that the listening device (iOS devices) cannot connect to the device - As such, like you said, the client responds by sending an HTTP request to the IP address in the BLE advertisement.

@connorh315 Confirming your observation that simpleBLE doesnt do advertising, I found this discussion on the simpleBLE discord server: It also seems that simpleBLE development was previously supported by developer's previous employer, and has now stalled (he comments: For a project like this to prosper, raw manpower is not enough if it’s not coordinated, as it leads to a steady degradation of the code quality. Either by managing people or coding myself, this is work that needs to put food on the table in one way or another.)

covertagent
—
08/09/2023 10:19 AM
Hey Folks, Can SimpleBle enable Bluetooth and put the device in discovery mode? As far as I could check, there are samples to connect, list devices etc but I couldn't find any samples that can just enable the Bluetooth and put it in discovery mode. I found Bluetooth client-specific functionalities in this library but not the server side functionalities. Can anyone please help here?
ER!K
—
08/09/2023 10:37 AM
What would "discovery mode" refer to? A BLE peripheral doing advertising would be considered discoverable as long as the flags field of the advertisement payload is set accordingly. This would be purely done on the peripheral side and hence not related to SimpleBLE doing the central (scanning) role on a PC. SimpleBLE (currently) only supports the central role, it cannot do advertising or expose a GATT table or other peripheral functions.
covertagent
—
08/09/2023 10:50 AM
Thanks for the prompt response @er!K, Noob question, Do you know of any libraries that can do advertising?
ER!K
—
08/09/2023 10:55 AM
You could check https://github.com/kevincar/bless being a Python library (was kind of derived from Bleak). Did not use it myself yet, so no clues whether good or not.

I think that BLE on Windows is going to be a bit of a pain to implement, I can't find any reliable docs/examples on using the Bluetooth libraries to broadcast an advertisement. Perhaps for now, I could just make the functions empty when building for Windows? It's a bit of a cheap cop out but I don't have long before I go back to work and don't want to leave this open - All this means is that the feature will only work on Linux. What do we all think?

That's fine.

If its activated by an option (e.g. -ble ) one can test for windows and reject the option with an error message at startup.
Just put some comment explaining what would be needed to get it working on windows for possible future use.

I am more worried about your earlier mention of running as root, etc.

can you explain a bit more ?

is this a useful ref?

https://punchthrough.com/creating-a-ble-peripheral-with-bluez/

ideally Uxplay would just write the advertisement that should be sent to a file that only exists while uxplay is running,and some bluez service (running as root) would detect that file, read it and transmit the contents with BLE. Is such a thing possible?

I've had a look at the link that you've sent, I am probably able to rewrite it all to use the D-Bus, however I'm not entirely sure that this circumvents using sudo. I could give it a go though. As an alternative, I have discovered that a user can run sudo setcap 'cap_net_raw,cap_net_admin=eip' ./uxplay and that will grant the necessary BTLE privileges to the binary. Additionally, I believe that the user may be able to add their user account to the bluetooth group to achieve a similar result.

Let me know whether this is suitable, as I do understand your concern with running UxPlay as sudo (especially since it seems to break GStreamer).

once a working prototype is available, one can see how to improve it.

Ideally uxplay would write a file with the BLE message to be sent, just like the -dacp option, and a separate helper script would periodically check for the existence of that file and if it is found send the BLE message.

UxPlay works fine as root, with the firewall switched off, I checked. This is fine for testing.

The main thing is to have a working prototype to start with.

What's the problem with the current prototype so I know what needs to be done to improve that atm? And just so I understand as well what's the reason you want a separate helper script rather than the solution being a part of the UxPlay binary?

BLE is tangential to UxPlay and not really core.

I think it would make sense to use something like python for ble as it already has a (probably) better package (bleak).

Also, I am going to say with some confidence that ble will eventually run into issues like needing to be rebroadcasted, restarted, what is Bluetooth is turned of, etc. let a user script take care of that. It also allows faster iteration and the feature can be published quicker. Dont want to use python? Any language can read / watch a file

@connorh315 I haven't yet tested your prototype, if its working, that's great as a start.

but writing the platform-independent BLE message text to a file that something else reads and sends the message is a truly cross-platform solution.

@connorh315 I am trying to test your prototype.

1. avahi is tricky to suppress: use "sudo systemctl mask avahi-daemon" (use "unmask" then "enable" and "start" to restore it);

2. uxplay now fails to start because there is no DNS_SD service. comment out "return -3" at (your uxplay.cpp line 1299) and "return -4" at line 1305.

This means uxplay -btip will run with no avahi-daemon service present. This should test your BLE advertisement.

3. Please confirm that your BLE works for you in the absence of DNS_SD service.

4. My iPad now does not see any UxPlay service when "uxplay -btip" is run (as root). What should I see? What precise things should I do?

• Please give complete instructions!

Sure. Current setup:
Receiver: Raspberry Pi 4
Sender: Doesn't matter but iPad Pro M1, iPhone 15, MacBook Air M1

In order to suppress dnssd, I added a return 1 on line 380 of dnssd.c to stop the service from being registered. That should be enough to test that it is the BLE that is driving the discovery.

One issue that is definitely something that I need to fix is that the IP that it grabs is from the eth0 interface which some people will not be using as they'll be on wifi instead.

I'd suggest adding a logger on line 90 of ble.c that logs the value of the ip_address variable to see which IP address it is broadcasting. I'll have a look at fixing this when I get a moment.

Should the code fail to initialize/send commands to the bluetooth interface it will spit out two messages, the first message being either line 49 or line 55. The second message is in uxplay.cpp where it will output that it will disable all attempts at BLE for the rest of the lifetime of the program (to prevent the console from being spammed).

Should all be well, you won't see any messages in the console regarding bluetooth (perhaps I should change this and make it clear the service is running?)

describe what I should see on the client: I assume uxplay should show up as a choice for mirroring (but for me it doesn't, with dnssd disabled)?

what tests should show ble is working? (something with bluetoothctl, I guess).

So if its working, with dnssd suppressed, I should see uxplay on the client in the normal way?

should I also use bluetoothctl to verify BLE in some way?

• [aside]. I see you are also working on encryption/decryption (chachapoly). I did all the work to implement SRP6a, (for non-transient pairing) but missed that the final pair-verify response from the server needs to send some code to trigger encryption. This would be a useful contribution. are you basing it on pair_ap? (we use OpenSSL not libsodium, pair_ap allows either)

This would be a very useful contribution.

https://github.com/FDH2/UxPlay/wiki/crypto

So if its working, with dnssd suppressed, I should see uxplay on the client in the normal way?

Exactly. I think I need to have a go at fixing some of the issues I've already described and then I'll set up a few other systems to test, would you mind telling me what system you're testing this on - Or at the very least the distro and the bluetooth card (especially what type it is: USB, internal etc.).

should I also use bluetoothctl to verify BLE in some way?

Assuming no errors are displayed it should have correctly initialized and set up the HCI for advertising. Unfortunately there is no way of viewing the advertisements that are sent out (Unless you're willing to buy a BLE development board like I did) in Wireshark.

[aside]. I see you are also working on encryption/decryption (chachapoly). I did all the work to implement SRP6a, (for non-transient pairing) but missed that the final pair-verify response from the server needs to send some code to trigger encryption. This would be a useful contribution. are you basing it on pair_ap? (we use OpenSSL not libsodium, pair_ap allows either)

Transient pairing does not require pair-verify at all, once the initial M1-M4 steps are completed the devices can derive their public/private keys off of the session_key and at that point begin encrypting their comms. I initially added libsodium I was planning on including the entire pair_ap repo as a submodule, but there was a lot of build errors that I couldn't wrap my head around so I ended up moving the individual functions myself. Despite it being in the Cmake file, I didn't use libsodium at all in the end.

I have two usb bluetooth v5 plugins testing on linux x86_64 desktop, varius distros, or Rpi 4 or 5. all with with systemd and bluetoothctl.

Yes, no libsodium please.

The main thing is to have a working prototype. Then a cleaned-up integration into uxplay can easily be done.

can you point out in your code the server response that starts the client encryption?

I have two usb bluetooth v5 plugins testing on linux x86_64 desktop, varius distros, or Rpi 4 or 5. all with with systemd and bluetoothctl.

Perfect, will try to fix up the code a little bit better. I assume you're connected to the network via wifi?

can you point out in your code the server response that starts the client encryption?

In raop_handler_pairsetup the case 0x03: is the M3 message from the client. The response to that message (M4) is the final unencrypted message that gets sent before the client begins sending encrypted messages (and will expect the server to follow with encrypted responses).

This might be a useful link:

https://medium.com/byborg-engineering/experimenting-with-web-bluetooth-advertisement-packets-662c7090e07b
https://reelyactive.github.io/diy/best-practices-ble-identifiers/

@connorh315 Here is ancient 10 years old code to apparently implement an BLE beacon (which I think is what you are doing?)) https://github.com/dburr/linux-ibeacon
It would have to run a root to use the BT, I'm not saying use this old code, just that it shows it can be done, apparently.

• I really like the idea of finding/building some script (python) that searches for a file written by uxplay that contains the advertisement bytes and broadcasts then so UxPlay does not have to run as root.

• I see your code getting to the BLE part, but dont see any announcement on the iPad. Should there be anything in the system log that will show something happening when uxplay send the bluetooth signal?

• I have ordered a nRF52840 dongle that can apparently be used as a cheap BLE sniffer .

@connorh315 Here is ancient 10 years old code to apparently implement an BLE beacon (which I think is what you are doing?)) https://github.com/dburr/linux-ibeacon It would have to run a root to use the BT, I'm not saying use this old code, just that it shows it can be done, apparently.

• I really like the idea of finding/building some script (python) that searches for a file written by uxplay that contains the advertisement bytes and broadcasts then so UxPlay does not have to run as root.
• I see your code getting to the BLE part, but dont see any announcement on the iPad. Should there be anything in the system log that will show something happening when uxplay send the bluetooth signal?
• I have ordered a nRF52840 dongle that can apparently be used as a cheap BLE sniffer .

• Interesting, will see if I can find any cross-platform solutions that will work as to simplify the process of making a Python script instead.

• I'm taking a look into this now, will add a debug message that says something along the lines of BLE: Advertising IP address xxx.xxx.xxx.xxx

• That's exactly what I use to sniff the packets. Since you're probably going to use wireshark the best thing you can do is use this filter: frame.len == 47 (the frame read size is constant and always 47). Just to save you some time, if you try to sniff out the packets from a real MacBook/Apple TV (HD Model and above) you're not going to find the packets by looking at the sending address as Apple intentionally use random BLE mac addresses to avoid tracking. However, you'll know if you're looking at the right packets as the payload will be exactly 8 bytes as I believe I described above (09 06 03 etc. etc.).

Question for you though, how can I get the IPV4 address of the adapter that the http server hosts a socket on? Is there a struct containing this as I couldn't easily find one by search. That's probably the main reason why the code wasn't working for you as no matter what it's currently broadcasting the address of the ethernet interface, and not the actual interface used for the http server.

• It looks like systemd has a service exactly for monitoring files for changes in existence or content.
  https://www.linux.com/topic/desktop/systemd-services-monitoring-files-and-directories

• this code in uxplay.cpp finds the MAC address of the active network card. I think it can also find the address.

 static std::string find_mac () {
/*  finds the MAC address of a network interface *
 *  in a Windows, Linux, *BSD or macOS system.   */

The address being used in the test is in fact eth0: so your code is using the correct address , so that doesnt explain why it didnt work. I will test with a bluetooth 4.0 adapter which I also ordered , (I'm using 5.x)

@connorh315

I'm working on having the uxplay startup extract ipv6, ipv6 (global) and ipv6 (link-local) non-temporary addresses for
the active network interface. I'm not sure which of the two ipv6 address should be used, or if they should only be used if there is no ipv4 address. It's a modification of find_mac().

Uxplay.c (github, master) will soon have std::strings ip4, ip6_global, ip6_local available soon after uxplay starts.

I'll post here when it's available.

Has to be an ipv4 address btw, thats the standard set for the ble advert

Has to be an ipv4 address btw, thats the standard set for the ble advert

That's a relief! I figured out with a lot of google help how to distinguish "permanent" from "temporary"(privacy) ipv6 addresses in Windows, but in Linux it was still not clear. In Linux "old style" permanent ipv6 addresses based on MAC address are still in use, "new style" ones are not so common, but I did not yet find how to detect these from C.

Has to be an ipv4 address btw, thats the standard set for the ble advert

That's a relief! I figured out with a lot of google help how to distinguish "permanent" from "temporary"(privacy) ipv6 addresses in Windows, but in Linux it was still not clear. In Linux "old style" permanent ipv6 addresses based on MAC address are still in use, "new style" ones are not so common, but I did not yet find how to detect these from C.

Gotcha. I think there is a variation in the BLE advertisement that I'd read somewhere (something along the lines of 09 06 06 instead of 09 06 03 for the message). I wasn't entirely sure what the purpose of it might be but interestingly that might be ipv6? Doesn't really matter though ipv4 should be suitable. Would you still like me to clean this up or would you like to test yourself first and then we clean this all up afterwards?

Unrelated but I figured out how to trigger a proper h265 stream that contains the necessary information to decode it. I posted about my findings in a discussion in this repo. Not sure if you'd seen it or not.

The nRF52840 arrived, but the setup instructions seem daunting. Does one need the whole SDK installed to just use the sniffer feature, or can one dispense with that.?

The nRF52840 arrived, but the setup instructions seem daunting. Does one need the whole SDK installed to just use the sniffer feature, or can one dispense with that.?

Err, don't think you need the SDK. I followed these instructions and it worked just fine. Something that's worth noting is I had to flash my dongle with new firmware for it to be set up in sniffing mode.

I think a custom library is going to need to be built for advertising over BLE on all the different operating systems. Additionally, I think that DBus can be used to circumvent needing root, as according to this root should only be required because I'm directly interfacing with hci, rather than using a higher level impl. such as DBus. Not too sure on that though.

sudo setcap 'cap_net_raw,cap_net_admin+eip' ./uxplay works for me. I don't think its actually working though. I blocked outgoing mdns firewall-cmd --direct --add-rule ipv4 filter OUTPUT 0 -p udp --dport=5353 -j DROP and ran uxplay with the -btle command and nothing showed up on.

https://stackoverflow.com/questions/37287026/bluez-library-access-as-non-root-user
https://www.ctrl.blog/entry/how-to-disable-mdns-linux.html

systemd guide:

https://www.linux.com/topic/desktop/systemd-services-monitoring-files-and-directories
https://www.linux.com/training-tutorials/systemd-services-reacting-change/

objective:

"uxplay -btip" when started writes a file ~/.uxplay.btip containing the BLE announcement. On exit, uxplay deletes this file. This can be modeled on the similar action of the -dacp option.

systemd monitors the existence of this file. while it exists, it runs a systemd service every x secs that uses bluez to send the uxplay BLE service advertisement.

After Bluetooth discovery, is Wi-Fi Direct networking still needed?

After Bluetooth discovery, is Wi-Fi Direct networking still needed?

If you mean needed in the sense of discovering the device: no. The Bluetooth discovery tells the sender what IP to find the receiver under. The sender will then attempt a connection under the local network it is attached to, and will discover the receiver if it is on the same network.

I do still think that Wi-Fi direct would be a nice feature though, so if you were asking on whether it is worth implementing Wi-Fi Direct: I still think so.

I misunderstood; I thought this Bluetooth was Bluetooth broadcasting in P2P.

https://support.apple.com/en-hk/guide/deployment/dep9151c4ace/web